Skip to content

fix(installer): execute-gate installed tools so a broken binary fails Step 1 (#411) - #451

Merged
shujaatTracebloc merged 6 commits into
developfrom
fix/411-execute-gate-tools
Jul 29, 2026
Merged

fix(installer): execute-gate installed tools so a broken binary fails Step 1 (#411)#451
shujaatTracebloc merged 6 commits into
developfrom
fix/411-execute-gate-tools

Conversation

@shujaatTracebloc

@shujaatTracebloc shujaatTracebloc commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Closes #411.

Problem

The only post-install "verification" was a log-only, failure-masking interpolation (Log "k3d: $(k3d version …)" / … 2>/dev/null || echo present). So a corrupt or wrong-architecture binary — winget shims / partial installs skip the direct path's checksum verify, and brew delivers with no checksum of ours (#429) — still printed ✔ System tools and only died at cluster-create in Step 2. Lukas's audit confirmed it's all three OSes.

Fix (both halves + macOS)

  • Shared execute-gate helpers: assert_tool_runs (common.sh, bash) and Assert-ToolRuns (install-k8s.ps1, PowerShell). Each runs the tool's self-check; on a non-zero exit or an exception it removes the located binary and fails loudly with an arch-aware remedy, so the tool step fails instead of the cluster step.
  • Gate kubectl / k3d / helm after install on Linux, macOS, and Windows — on both the fresh-install and already-present paths. System tools success only prints once all gates pass.
  • kubectl is gated with version --client, not --short (removed in kubectl 1.28+, which would false-fail the gate).
  • New check-drift.sh parity check (_drift_execute_gates) so no installer can silently drop a gate.

Acceptance criteria

  • A corrupt / wrong-arch k3d.exe fails Step 1 loudly with an actionable message; Step 2 is never reached (Err/error exits before it).
  • Coverage for the failure path — widened to bats + Pester + the drift check.

Tests (+11)

  • common.bats +2 (working tool passes & binary untouched / broken tool errors + removes the binary)
  • check-drift.bats +2 (execute-gate parity)
  • install-k8s.Tests.ps1 +7 Pester (exit-nonzero / exception / binary-removal / arch remedy / static gates)

Verified locally (CI-identical):

  • Pester full: 182 passed, 0 failed, 8 skipped (Windows-only)
  • all bats: 3 failures are pre-existing macOS-local flakes (my new tests all pass); ubuntu CI is the arbiter
  • PSScriptAnalyzer: 0 Error-severity · check-style / drift: clean · manifest regenerated

Note

Low Risk
Changes are confined to installer verification and drift tests; behavior is fail-fast with guarded binary removal, with no auth or runtime cluster logic touched.

Overview
Replaces log-only post-install version checks (which could still show ✔ System tools when a binary was corrupt or wrong-arch) with real execute gates that run each tool’s version self-check and fail Step 1 with an arch-aware message before cluster creation.

Adds assert_tool_runs in common.sh and Assert-ToolRuns in install-k8s.ps1, wired for kubectl (version --client), k3d, and helm (bare version, not --short) on Linux, macOS, and Windows for both fresh installs and already-on-PATH tools. Optional --rm / -BinPath removes only the installer-placed binary when it is the one that failed, so re-runs can self-heal without deleting winget/brew copies elsewhere on PATH.

Adds _drift_execute_gates in check-drift.sh plus bats/Pester coverage so no platform can drop a gate silently; updates manifest.sha256 and Linux setup test mocks so gates don’t break install-path assertions.

Reviewed by Cursor Bugbot for commit f0f2626. Bugbot is set up for automated code reviews on this repo. Configure here.

… Step 1 (#411)

The only post-install "verification" was a log-only, failure-masking
interpolation, so a corrupt or wrong-architecture binary (winget shims /
partial installs skip checksum verify; brew delivers with no checksum of ours)
still printed "System tools" and only died at cluster-create in Step 2.
All three OSes had the gap.

- New shared execute-gate helpers: assert_tool_runs (common.sh, bash) and
  Assert-ToolRuns (install-k8s.ps1, PowerShell). Each runs the tool's self-check;
  on non-zero exit or an exception it removes the located binary and fails loudly
  with an arch-aware remedy, so the tool step fails instead of the cluster step.
- Gate kubectl / k3d / helm after install on Linux (setup-linux.sh), macOS
  (setup-macos.sh), and Windows (install-k8s.ps1), on both the fresh-install and
  already-present paths. "System tools" success only prints once all gates pass.
- kubectl is gated with `version --client` (NOT --short — removed in kubectl
  1.28+, which would false-fail the gate).
- New check-drift.sh parity check (_drift_execute_gates) so no installer can
  silently drop a gate for a tool.

Tests: +2 bats (common — working tool passes / broken tool errors + removes the
binary), +2 bats (drift parity), +7 Pester (exit-nonzero / exception /
binary-removal / arch remedy / static gates).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTracebloc shujaatTracebloc self-assigned this Jul 28, 2026
…-gate (#411)

The execute-gate (#411) runs `<tool> version` after install, but the
setup-linux.bats harness only marked tools "present" via has() — it never
provided a RUNNABLE k3d/kubectl/helm. On the toolless CI runner the gate's
probe found no binary and failed 4 install_k3d tests; locally it false-passed
because a real k3d was on PATH. Add silent (non-recording) runnable stubs to
setup() so the gate has something to execute and they shadow any real tool on a
dev host. Test-only; no production change.
…te-tools

# Conflicts:
#	scripts/manifest.sha256
#	scripts/tests/install-k8s.Tests.ps1
…te-tools

# Conflicts:
#	scripts/manifest.sha256
#	scripts/tests/check-drift.bats
#	scripts/tests/check-drift.sh
#	scripts/tests/install-k8s.Tests.ps1
@shujaatTracebloc
shujaatTracebloc marked this pull request as ready for review July 28, 2026 15:40
Comment thread scripts/tests/check-drift.sh Outdated
Comment thread scripts/install-k8s.ps1 Outdated
Comment thread scripts/lib/common.sh Outdated
Comment thread scripts/lib/setup-macos.sh Outdated
Comment thread scripts/install-k8s.ps1
…on; drift extracts calls (review #411)

Three review findings on #411 (saadqbal + Bugbot):

- Execute-gate deleted the wrong binary on the already-present path. Removal is now
  OPT-IN via `assert_tool_runs --rm <path>` / PS `-BinPath`, passed ONLY by the
  fresh-install callers that placed the binary. On the present / brew / winget path
  we don't pass it, so a broken pkg-managed binary is left in place (deleting a brew
  symlink just wedges the re-run, and the bad copy may be elsewhere on PATH). bash:
  --rm on Linux fresh-install, none on present or macOS/brew. ps1: -BinPath tracks
  the direct-download dest ($null for winget/present).
- helm was gated with `version --short`; --short can be dropped like kubectl's was,
  which would false-fail the gate and (previously) delete a good binary. Use bare
  `helm version` on all three OSes.
- _drift_execute_gates used a whole-file grep that could match comments. It now
  strips comment lines and matches the actual call (`assert_tool_runs … <t> version`
  / `Assert-ToolRuns -Name "<t>"`), handling the new --rm form. (no grep -q under
  pipefail — SIGPIPE would false-fail.)

Tests: common.bats split into --rm-removes vs no-rm-leaves; check-drift.bats adds
--rm-form + commented-out-gate cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3dc866f. Configure here.

Comment thread scripts/lib/setup-linux.sh Outdated
…d it AND it ran (Bugbot #411 r2)

Round-1 made removal opt-in but only on the fresh-install path, so a broken
installer-placed kubectl/k3d on the ALREADY-PRESENT path (left by a prior run)
stayed → `has` true → re-run couldn't self-heal (Bugbot). helm already handled it.

Unify all three: callers pass --rm "$TB_TOOLS_DIR/<tool>" (bash) / -BinPath
"$TOOL_DIR\<tool>.exe" (ps1) on EVERY path, and the gate removes that path only
when the binary that actually ran resolves to it — `command -v … -ef <path>` on
bash, `(Get-Command).Source -eq $BinPath` on ps1. So a broken copy we own (fresh
or prior-run) self-heals, while a brew/winget/pkg-manager copy elsewhere on PATH
is never touched (satisfies the earlier reviewer guard too). Dropped the now-moot
$k3dDest/$helmDest null-tracking and helm's -f branch.

Tests: common.bats +decoy-copy case; install-k8s.Tests.ps1 removal split into
ran-here (removed) vs resolved-elsewhere (left); check-drift already covers --rm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@saadqbal
saadqbal self-requested a review July 29, 2026 06:20
@shujaatTracebloc
shujaatTracebloc merged commit 8b1daa5 into develop Jul 29, 2026
31 checks passed
@shujaatTracebloc
shujaatTracebloc deleted the fix/411-execute-gate-tools branch July 29, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants